Skip to content

Main Graph v2#6173

Open
RobertJoonas wants to merge 30 commits intomasterfrom
main-graph-v2-backend
Open

Main Graph v2#6173
RobertJoonas wants to merge 30 commits intomasterfrom
main-graph-v2-backend

Conversation

@RobertJoonas
Copy link
Copy Markdown
Contributor

@RobertJoonas RobertJoonas commented Mar 18, 2026

Changes

This PR gets rid of the /main-graph endpoint in favor of the new API v2 based /query endpoint, and builds a whole new hand-rolled graph with D3.

Backend: This PR -- see description below
Frontend: #6159 (which has now been merged into this one)

1. Change the result format of comparison results (when time dimension breakdown)

With this PR, the query API starts returning comparison_results as a separate response field if dimensions == [time:*], while breakdown and aggregate responses keep comparisons nested for each row in the original results list.

In hindsight, the new comparison results format could have been avoided by filling in all the missing buckets instead. This would have let us avoid returning a bunch of meta fields (time_labels, comparison_time_labels, time_label_indices, comparison_time_label_indices, and empty_metrics).

The main reason I went with this approach was to prepare comparisons for a public API release (playing well with time_labels), but since it’s still far from perfect (a noisy query.include + meta; difficult for the client to read the data), we’ll still need to re-evaluate this. We should also carefully consider what to do with include.total_rows and multi-dimensional (time + non-time) breakdowns, where zipping comparison <> main results gets tricky.

2. Add new response.meta and query.include fields

Note: all these flags are internal-use only. Nothing gets released into the public API

  • meta.comparison_time_labels
    • same as meta.time_labels but for generating the comparison plot
    • added when include.time_labels && include.compare
  • meta.present_index
    • index of the time label that is currently ongoing
    • added when include.present_index
    • NOTE: with the recent decision to change the dotted line meaning to "any incomplete bucket", this might not be necessary. However, I would keep this around until we're certain that the new definition of the dotted line is OK.
  • meta.time_label_result_indices
    • a list (of the same length as meta.time_labels) of indices where each item corresponds to an index in the results list or null when there is no result for that bucket. Makes it much easier for the frontend to understand the data.
    • added when include.time_label_result_indices
  • meta.comparison_time_label_result_indices
    • same as meta.time_label_result_indices but for comparison results
    • added when include.time_label_result_indices && include.compare
  • meta.partial_time_labels
    • a list of length 0, 1, or 2 where items correspond to labels of time buckets that are incomplete. Only the first and last can be incomplete, which is why the max length is 2.
    • added when include.partial_time_labels
  • meta.comparison_partial_time_labels
    • same as meta.partial_time_labels but for comparison results
    • added when include.partial_time_labels && include.compare
  • meta.empty_metrics
    • a list representing an "empty value" for every queried metric (in the same order). E.g. when query.metrics = ['visitors', 'time_on_page'], then meta.empty_metrics = [0, null]. Makes it possible for the frontend to fill in missing buckets without re-defining empty metrics.
    • added when include.empty_metrics && "event:goal" not in query.dimensions - there shouldn't be any need to ask for empty metrics in the goal breakdown, but this makes it clear that there's a conflict (possibly multiple revenue currencies).

3. Remove old code

  • Delete the main-graph endpoint
  • Remove comparisons logic from legacy timeseries (see commit with description 7ad120a)

4. Transform main_graph_test.exs

A major rewrite of main_graph_test.exs - populate_stats and other tests setup remained the same, but the request is now made against the new endpoint and the assertions on the returned data structures have changed significantly.

Tests

  • Automated tests have been added

Changelog

  • Entry has been added to changelog

Documentation

  • This change does not need a documentation update

Dark mode

  • The UI has been tested both in dark and light mode

* Add a new response structure: a separate `comparison_results` field,
  used only when comparing in a time-dimensional query.

* Add `meta.time_label_result_indices` to response (exclusive to internal
  API). Makes it easier for FE to find buckets for time labels.
This is currently only used by the main graph, which is going to move to
a new endpoint in this PR.

* The CSV export is currently ignoring comparisons

* the `&compare=previous_period` option in Stats API v1 is ignored by
  the timeseries endpoint
@RobertJoonas RobertJoonas marked this pull request as draft March 18, 2026 12:56
@apata apata mentioned this pull request Apr 7, 2026
10 tasks
@RobertJoonas RobertJoonas requested a review from ukutaht April 14, 2026 08:51
end

def present_index(time_labels, query) do
now = DateTime.now!(query.timezone)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels like this should be using query.now as well (?)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would make sense, yes. I'll keep that in mind in case we end up actually using present_index on the FE. Otherwise, we'll remove the field.

* Add generic graph component and use it for main graph

* Remove superfluous mobile related code

* Add isTouchDevice state and touch-device specific zoom instructions

* Move useMainGraphWidth

* Center tooltip

* Adjust tooltip position

* Clarify pointerup

* Extract fetching main graph and remapping main graph data

* Unify BE and FE change calculations

* Stop wrapping change arrow, unify with ComparisonTooltipContent

* Exports

* Attempt partial logic

* Draw dashed partial periods at the start and end of main series

* Simplify

* Clarify series

* Better types

* Make remapAndFillData take accessors

* Get metric label using utility

* Refactor value and comparisonValue to numericValue and comparisonNumericValue

* Refactor outerValue and comparisoOuterValue to value and comparisonValue

* Fix format

* Fix type error in test

* No need to assert isPartial

* Fix issue with full crash when switching between periods while tooltip

* Add necessary types

* Handle comparison_partial_time_labels

* Handle [null] style result items

* Hint expected tick count to y axis nicing

* Remove old line graph

* Clarify graph inputs

* Add MetricValue type

* Extract logic to read first and last time labels

* Clarify remapAndFillData

* Stop requesting for present_index as it is unused

* Clarify getLineSegments comment and refactor getNumericValue arg

* Remove unnecessary guard and unnecessary return value

* Refactor x tick values function and the way it is passed around

* Add getXDomain function

* Add tests for some graph.ts utils

* Add constants to top of graph.tsx, move utiltiy types to bottom
@RobertJoonas RobertJoonas changed the title Main graph v2 backend Main Graph v2 Apr 15, 2026
@github-actions
Copy link
Copy Markdown

Preview environment👷🏼‍♀️🏗️
PR-6173

@RobertJoonas RobertJoonas marked this pull request as ready for review April 15, 2026 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants